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7 Multi-Function High-Speed Network Interface 

8 

9 Field of the Invention 

10 The current data communication invention is directed to 
fg a high speed network interface that is capable of the 

Ijy transmission and reception of multiple types of variable 

101 length data packets. In addition, the invention includes a 

Iffj media access control mechanism. Data packets can be sent 

1§ over a variety of physical media, including single mode 

lBl fiber, multi-mode fiber, and parallel fibers. 

\ y 

1&3 Background of the Invention 

19 In data communications networks, a large number of 

20 methods are used to encapsulate communication data packets 

21 such as OSI (Open Systems Interconnect) layer 3 TCP/IP 

22 packets for the purpose of transmission over local or layer 

23 2 networks and over specific point to point layer 1 physical 

24 links. Examples of OSI layer 2 network encapsulation and 

25 transmission and access control methods include 10Mb 
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1 Ethernet, IOOMd Ethernet, gigabit Ethernet (IEEE 802. 3z) , 

2 IEEE 802. 1Q, IEEE 802.3k, FDDI (ANSI X3T9.5), and token ring 

3 (IEEE 802.5), There are also a large number of methods to 

4 encapsulate layer 3 packets for transmission over layer 2 

5 networks. For point to point links, available encapsulations 

6 include PPP over HDLC (RFC16 61) , and Packet over Sonet (IETF 

7 RFC1619) . 

8 In a generic Ethernet packet, the header information 

9 that is immediately needed for a switching decision is the 
10 layer 2 media access control (MAC) source and destination 
1LH addresses and, optional 802. 1Q tag information. For an IP 
12[y packet, routing information is contained in the IP source 
13Q and destination addresses. The MAC source and destination 
I4ff\ addresses are used for layer 2 switching, wherein the 

15?. destination address is matched with the port having 

1601 previously received a source address of the same value. The 

17 Lm layer 2 source and destination information is readily 

18 Q available in the first 12 bytes of the Ethernet packet, and 

19 generally presents no challenge in extraction. Higher level 

20 layer 3 Internet Protocol (IP), and other types of protocol 

21 packets present somewhat greater difficulty. For IP, a 32 

22 bit IP source and 32 bit IP destination address are needed 

23 for the routing decision, and the hardware must go through a 

24 decision tree to determine what type of packet is being 

25 examined, what protocol type it is, and thereafter extract 



Patent Application for High Speed Network Interface by Bechtolsheim, Frazier, and Edsall 2 
File: C:\patent\cisco\13161_MF_Hi_Speed_Int\13161patent.doc Last saved 06/24/99 12:05 AM 



1 addressing information. Virtual Local Area Network (VLAN) 

2 information may be added indicating which VLAN the packet 

3 has membership. Each switch keeps a copy of a table with 

4 the VLAN value associated with a port of exit. When a 

5 packet bearing this VLAN value arrives, a hardware-based 

6 lookup is performed into a table, which yields the 

7 associated port of exit for the packet. If the decision 

8 tree determines that the packet is not of a supported type, 

9 then the treatment reverts back to the layer 2 switching 
10 treatment of a simple Ethernet packet, and the time spent 
if examining the packet is lost. Additionally, for each 

fe; separate protocol, such as IP, IPX, Appletalk, etc., this 

W examination of the packet must occur, and the information 

fi; will appear in different places and formats in the packet. 

E5L It would be useful to have a single method of access for the 

4£ transmission and reception of switching and routing 

tti information for such data packets including a common header 

t& format . 
19 

20 Objects of the Invention 

21 A first object of the invention is to provide a packet 

22 encapsulation that includes a start symbol and a type field 

23 which provides for the encapsulation of a variety of data 

24 packet types, including Ethernet, FDDI, Token Ring, ATM 

25 cells, and others. 
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A second object of the invention is a packet 
encapsulation that provides for layer 2 prioritization and 
virtual LAN information for any type of data packet. 

A third object of the invention is the provision of a 
header field which allows application specific packet 
information in addition to the payload. 

A fourth object of the invention is the provision of a 
CRC that includes the header and payload of the packet. 

A fifth object of the invention is to provide for data 
transmission at a variety of speeds. 

A sixth object of the invention is the provision of a 
flow control mechanism to the media as indicated by the 
receiver . 

A seventh object of the invention is to allow data 
transmission in a variety of encodings in a transmission and 
reception media, including a serial channel, as well as any 
number of parallel channels. 

Summary of the Invention 

The present invention comprises a method for encoding 
and decoding data referred to as GX packets, a transmit 
processor, and a receive processor. The transmit processor 
includes a transmit buffer/controller dividing the data into 
a plurality of transmit data lanes, a plurality of transmit 
encoders, each encoder accepting information from a unique 

Patent Application for High Speed Network Interface by Bechtolsheim, Frazier, and Edsall 4 
File: C:\patent\cisco\13161_MF_Hi_SpeedJnt\13161patent.doc Last saved 06/24/99 12:05 AM 



1 transmit data lane and producing an output stream of clock- 

2 encoded data, and a plurality of transmit serializers, each 

3 accepting a unique stream of transmit encoded data, and 

4 serially encoding it for transmission. A variable-speed 

5 transmit clock circuit affords clocking of the elements of 

6 the transmit processor. The transmit encoders add clock 

7 recovery information to the data stream, optionally using 

8 the 8B/10B encoding method, and also insert START , END, 

9 I DL ELEVEN, IDLE_ODD, IDLE_EVENJBUSY, and IDLE_ODDJBUSY 

10 information as instructed by the input buffer/controller. 

\% The receive processor comprises a plurality of receive 

f£ deserializers, each receiving a serial stream of encoded 

'i% data, and producing a stream of parallel encoded data. 

ft These encoded parallel streams are passed on to a plurality 

K of receive decoders, each of which decodes a stream of 

IjS encoded data into a stream of byte information, as well as 

'¥i decoding control information such as START, END, IDLE_EVEN, 

W IDLE_0DD, 1DLE_EVEN_BUSY, and IDLE_ODD_BUSY . The receive 

19 processor/controller accepts these streams of data and 

20 control signals, organizes the recovered byte streams into 

21 recovered packets, and also reports errors associated with 

22 the data recovery process. A recovered receive clock for 

23 each data lane is used to clock synchronized data into the 

24 elasticity buffer of the receive processor. 
25 
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1 

2 Brief Description of the Drawings 

3 Figure 1 is a prior art IEEE 802.3 Ethernet Packet 

4 including an IP payload. 

5 Figure 2 is a prior art ATM cell. 

6 Figure 3a shows the GX packet format with header 

7 details. 

8 Figure 3b shows the header for the GX packet. 

9 Figure 4 shows a data stream including packets and 
10 inter-packet intervals. 

fjj Figure 5 shows the GX packet format with payload 

Kg details. 

ft Figure 6 shows the case where the GX payload is an 

fK Ethernet packet . 

Figure 7 shows the case where the GX payload is a 

{6 native IP packet. 

t% Figure 8 shows the case where the GX payload is an ATM 

f£ cell. 

19 Figure 9 shows the case where the GX payload is an FDDI 

20 packet . 

21 Figure 10 shows the case where the GX payload is a 

22 Token Ring packet . 

23 Figure 11 shows the GX packet format where the data is 

24 transmitted and received across 8 data lanes . 
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1 Figure 12 shows the GX packet format where the data is 

2 transmitted and received across 4 data lanes. 

3 Figure 13 shows the GX packet format where the data is 

4 transmitted and received across 2 data lanes. 

5 Figure 14 shows the GX packet format where the data is 

6 transmitted and received across 1 data lane. 

7 Figure 15 is an 8 data lane transmit processor for the 

8 packet of figure 11. 

9 Figure 16 is a 4 data lane transmit processor for the 
10 packet of figure 12 . 

fiij Figure 17a shows the 8B/10B encoder. 

f?| Figure 17b shows the encoding scheme for the encoder of 

fi figure 17a. 

ff| Figure 18 is an 8 data lane receive processor for the 

packet of figure 11. 
fcj Figure 19 is a 4 data lane receive processor for figure 

12 . 

f§ ; Figure 2 0a shows the 10B/8B decoder. 

19 Figure 2 0b shows the decoding scheme for the decoder of 



20 figure 2 0a. 

21 

22 

23 Detailed Description of the Invention 

24 Figure 1 shows a prior art layer 2 Ethernet Packet 10, 

25 comprising a preamble 12, a header 16 comprising a MAC 
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1 Destination Address 14, a MAC Source Address 18, and 

2 length/type information 22. Payload 24 is followed by the 

3 Frame Check Sequence 26 which is a Cyclical Redundancy Check 

4 (CRC) polynomial applied over the entire Ethernet header 16 

5 and payload 24. For a generic layer 2 Ethernet packet, 

6 payload 2 4 contains the data. In the case of a layer 3 

7 protocol such as IP, payload 24 is arranged to further 

8 comprise an IP header 28, an IP source address 30, and an IP 

9 destination address 32, followed by the IP data 34. Other 

10 layer 3 protocols such as Appletalk, IPX, and the like have 

11 alternate arrangements for payload 24, but in general carry 

12 a layer-related source and destination address observed by 

13 the particular protocol. Other attributes of Ethernet 

14 packet 10 include variable length payload 24, which may vary 

15 from 46 bytes to 1500 bytes, as defined in the MAC layer 

16 specification IEEE 802.3. The general attributes of prior 

17 art packets as described in figure 1 include both layer 2 

18 (MAC) and layer 3 (network) source and destination 

19 addresses, and variable length data 24. These are used 

20 individually, or in combination, by network switches and 

21 routers to forward packets across layer 2 subnets, and 

22 represents information for which the switching hardware 

23 generally needs immediate access for the switching/routing 

24 decision. 
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1 Figure 2 shows an ATM cell 40 generally used in an ATM 

2 switching network. ATM networks set up end-to-end 

3 connections according to ATM Forum User Network Interface 

4 (UNI 3.1) protocols prior to the transmission of actual 

5 network data. The characteristic of ATM cells is a fixed 53 

6 byte cell comprising a 5 byte ATM header 31 and a 48 byte 

7 payload 50. The header contains an 8 bit VPI (virtual path 

8 index) 42 and a 16 bit VCI (virtual circuit index) 44; which 

9 is locally assigned and kept as an index into a look-up 

10 table which associates an exit port with this index. 

11 Figure 3a shows the GX packet format for the present 

12 invention. GX packet 52 comprises a GX header 54, a GX 

13 payload 56, and a Frame Check Sequence (FCS) 58, which 

14 operates over the entire GX header 54 and GX payload 5 6 

15 Figure 3b shows the details of the GX header of figure 

16 3a. GX header 54 includes a START symbol 60. BPDU field 62 

17 is a Bridge Protocol Data Unit field, which flags the 

18 following data as configuration information used by the 

19 spanning tree algorithms such as IEEE 802. ID, or other 

20 configuration information which needs to be given higher 

21 priority against packet loss during times of network 

22 congestion. Packet type field 64 indicates the type of GX 

23 payload data, such as ATM cell, Ethernet, FDD I , etc. VLAN 

24 field 66 contains VLAN information, PRIORITY field 67 

25 indicates priority information, and Application Specific 
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1 field 68 allows for the optional transmission of information 

2 specific to the needs of the data communication system. 

3 Figure 4 shows a GX data stream 70 comprising a 

4 plurality of GX packets 52, each followed by an inter-packet 

5 interval 74 which comprises an END 59 symbol and a variable 

6 amount of IDLE 72 time. The END symbol 59 immediately 

7 follows the GX FCS 58 of the preceding GX packet 52, and the 

8 variable IDLE 72 time allows for the continuous 

9 synchronization of the receiver during times when data is 
10 not being transmitted. 

Jj3 11 Figure 5 shows examples of different GX payload 56 

y 12 formats, illustrated by the figures 6 through 10. 

3 13 Figure 6 shows the GX payload 56 as a prior art 

hi 14 Ethernet packet, comprising Ethernet header 90, and Ethernet 

p 15 payload 92. The Ethernet FCS 26 is not transmitted, since 

Jy 16 the GX FCS 58 serves this error checking function in GX 

^ 17 links . 

18 Figure 7 shows a native IP payload comprising an IP 

19 frame 98 having an IP header 100, IP source address 102, IP 

20 destination address 104, and IP data 106. 

21 Figure 8 shows the GX payload 5 6 comprising an ATM 

22 format 108 including a reserved field 110, ATM header 112, 

23 and ATM payload 114. The optional reserved field 110 may 

24 occur before the ATM header 112 or after the ATM payload 

25 114. 
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1 Figure 9 shows the GX payload 5 6 comprising an FDDI 

2 format 116, including FDDI header 118, and FDDI payload 128. 

3 Figure 10 shows the GX payload 5 6 comprising a Token 

4 Ring format 13 0 including a Token Ring header 13 2 and Token 

5 Ring payload 142 . 

6 Each of the GX payloads 56 as described in figures 

7 6,8,9, and 10 is associated with a particular GX header type 

8 field value 64. The objective of field value 64 is to 

9 provide knowledge of the type and format of the associated 

10 GX payload data. It is clear to one skilled in the art that 

11 it is possible to add support for additional payload format 

12 types through the assignment of header type 64 beyond those 

13 discussed here, and the use of specific GX payload types 

14 described herein are not intended to limit the application 

15 of field types to only these examples used for illustration. 

16 Figure 11 shows the GX packet format for the case where 

17 the number of data lanes 152 n = 8. Data from the GX packet 

18 52 is distributed across 8 data lanes identified by the 

19 columns 0 through 7 of 152. Time intervals are identified 

20 alternately as odd (o) and even (e) as shown in column 154. 

21 K symbols are sent during even cycles, and R symbols sent 

22 during odd cycles, as represented by rows 15 6 and 158, 

23 respectively. This alternating sequence continues during 

24 inter-packet idle time 167, serving as the variable length 

25 inter-packet interval 74. The GX packet 168 comprises GX 
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1 header 164 and GX payload, starting at row 16 6, and ending 

2 at row 170. Data is instantaneously transmitted from column 

3 0 through column 7, row by row, until the final 4 byte FCS 

4 58 is sent, as noted by F0, Fl, F2 , and F3 in row 170, which 

5 is the end of the GX packet 52. The T symbol of row 171 

6 indicates the end of transmission, and the remainder of the 

7 data lanes are filled with odd idle symbol R. The following 

8 Ethernet packet 180 of figure 11 comprises GX header 172 

9 followed by data, the FCS, and an END symbol in rows 173 

10 through 175. If the receiver for the link were busy, the 

11 BUSY_IDLE symbols KB and RB would be sent during the inter- 

12 packet period 181 from END symbol T on column 7 of line 175 

13 followed by BUSY_IDLE symbols on line 176 through 179. 
14 

15 The algorithm used in transmission is as follows: 

16 1) Packet data is divided across n data lanes. 

17 2) Upon startup, each of data lanes 0 though n-1 sends 

18 idle symbols alternately coded K during even cycles, and R 

19 during odd cycles. The number of these initial startup idle 

20 packets is N id i e , and their purpose is to gain 

21 synchronization of the receiver prior to sending data. If 

22 the receiver of the link is busy, and unable to receive 

23 additional data, it signals this with the code KB during 

24 even cycles, and RB during odd cycles. 
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1 3) When a data packet is ready for transmission, a 

2 header is sent including the start symbol S on the first 

3 data lane, and remainder of the header on the remaining data 

4 lanes. The GX header includes information which declares 

5 the format of the packet data of the GX payload. 

6 4) Across all data lanes, data is sequentially sent up 

7 to, but not including, the last lane-wide data transfer. 

8 5) On the last lane-wide data transfer, the end of 

9 packet symbol T is sent on the data lane following the last 

10 valid data byte, and the balance of the data lanes is filled 

11 with K for even cycles, or R for odd cycles. If the 

12 receiver for the link is unable to receive new data, KB is 

13 sent during even cycles, and RB is sent during odd cycles. 

14 6) Until the availability of the next complete packet 

15 for transmission, the symbol K is sent during even cycles 

16 and R is sent during odd cycles across all data lanes. 

17 During the time the receiver is unable to receive new data, 

18 KB is sent during even cycles, and RB is sent during odd 

19 cycles. Additionally, a sequence of at least 32 bytes is 

20 sent at least once every n e i as ticity data bytes to accommodate 

21 clocking differences between systems. The value of n e i a sticit y 

22 is determined by the difference in clock frequency between 

23 systems, and is typically 2 15 bytes or greater. 

24 In the example of figure 11, a link is initialized with 

25 the transmission of a minimum number of idle patterns which 



Patent Application for High Speed Network Interface by Bechtolsheim, Frazier, and Edsall 13 
File: C:\patent\cisco\13 161„MF_Hi_Speed_Int\13 161patent.doc Last saved 06/24/99 12:05 AM 



1 are transmitted by all data lanes during even cycles as the 

2 symbol K and during odd cycles as the symbol R. This idle 

3 pattern continues a minimum time to enable symbol 

4 synchronization in the receiver. This enables the receiver 

5 to acquire lock, and the symbol decoders to operate for an 

6 interval on synchronization data, as well as to decode their 

7 separate version of odd/even, which the receiver will need 

8 for symbol decoding. In this example, a 64 byte Ethernet 

9 packet 168 is sent, comprising header 164, and data rows 166 

10 through 170, followed by END symbol T and IDLE_ODD R symbols 

11 of row 171. Since the last data symbol occurs on the final 

12 data lane of 170, data row 171 has the END symbol T sent on 

13 data lane 0, and the remainder of the data lanes carry the 

14 IDLE_ODD symbol R. The next packet sent is Ethernet packet 

15 180, comprising header 172, and data 173 through 175. In 

16 this case, the END symbol is sent on the last data lane 

17 during the last data transfer 175, which is data lane 7 for 

18 the present case of n=8 . 

19 Figure 12 shows the GX packet format for the case where 

20 the number of data lanes n = 4. As before, the data is 

21 presented to data lanes 192, and the frame comprises inter- 

22 packet idle pattern 224 which alternates during odd and even 

23 cycles of 196, 198, 200, and 202. The GX header is now sent 

24 over two cycles starting with the Start symbol in HO of 204, 

25 followed by the balance of the header in second cycle 206. 



Patent Application for High Speed Network Interface by Bechtolsheim, Frazier, and Edsall 14 
File: C:\patent\cisco\13161_MF_Hi_SpeedJnt\13161patent.doc Last saved 06/24/99 12:05 AM 



1 The GX payload 228 is sent from cycles 208 through FCS 216 

2 and END symbol T in line 218. Cycles 218 and 220 show a 

3 busy receiver, as BUS Y_ I DLE_E VEN KB and BUSY„IDLE_ODD RB are 

4 sent. The following cycle 222 signals an even cycle with 

5 the receiver no longer busy. 

6 Figure 13 shows the GX packet format for the case where 

7 the number of data lanes n=2 . For this case, the GX header 

8 and GX data are distributed across the available data lanes, 

9 as shown . 

10 Figure 14 shows the serial case of n=l, where only one 

11 data lane is used. As before, IDLE_0DD and IDLE_EVEN cycles 

12 are shown during the inter-packet idle interval 3 46, and the 

13 GX header is sent over 8 cycles as shown in 348, and the GX 

14 payload is sequentially sent from 320 through 340, followed 

15 by the END symbol T in row 342. 

16 It is clear to one skilled in the art that in the 

17 previous illustrations for various numbers of byte lanes 

18 that the assignment of a particular data byte stream to a 

19 particular byte lane is arbitrary, and the sequential 

20 assignment shown herein is for illustration only. 

21 Examining the header for each of the previous figures 

22 11, 12, 13, and 14, figure 3b shows the header format 

23 typically used by each of the previous examples. For the 

24 present example, where 8 data lanes are used, the header 54 

25 comprises the following bit fields: 
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1 



an 8 bit START symbol 60; 



2 



a single bit BPDU field 62; 



3 



a 7 bit type field 64 indicating the type of data 



4 carried (Ethernet, Token Ring, FDDI , ATM, IP, etc); 



5 



a 12 bit VLAN field; 



6 



a 3 bit priority field; 



7 



a 1 bit reserved field; ; 



8 a 32 bit application specific field 68 for future expansion. 

9 Alternatively, application specific field 68 may contain 

10 information that is specific to a particular protocol. The 

11 information in the GX header 54 is generally useful to the 

12 router or switch in the extraction of early information 

13 about the nature and handling of the information to be 

14 processed. The type field 64 is used to declare whether the 

15 payload data is Ethernet, ATM, or some other type, and the 

16 data that follows is used in a context specific to the type 

17 declaration 64. In the particular example, a BPDU bit 62 

18 can be set which ensures that BPDUs (Bridge Protocol Data 

19 Unit) used for Spanning Tree Protocol described in IEEE 

20 802. ID properly arrive. Often in networks that are 

21 congested because of a reconfiguration event (equipment that 

22 has recently failed, or is being added or removed) , BPDUs 

23 without such priority treatment and which are essential to 

24 the reconfiguration of the network are lost, or blocked from 
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1 transmission. By including this information in the header, 

2 they may be granted priority over ordinary data packets . 

3 Figure 15 shows a block diagram of the transmit 

4 processor 360. Input packets arrive via input interface 364 

5 to transmit buffer/controller 370 accompanied by 

6 control/data bits 366. Data is arranged in byte order 

7 across output ports 0 through n-1, shown for the instance 

8 where n=8 having data lanes 0 through 7 . In addition to 

9 data outputs, the transmit buffer/controller also outputs 

10 control information for each data lane. Each data lane is 

11 controlled by a single control signal such as 374a for lane 

12 0, and accompanied by transmit data such as 3 72a. During 

13 the times that control symbols are emitted, such as START, 

14 IDLE, IDLE_BUSY, and END, the control input is asserted 

15 true, and the associated 8 bit data indicates which special 

16 10B control symbol should be emitted. During the time that 

17 the control signal is not asserted, the incoming data is 

18 encoded using the prior art 8B/10B coding rule, as will be 

19 described later. When IDLE cells are to be output, the 

20 encoders across all data lanes are set to alternate between 

21 IDLE_EVEN and IDLE_ODD. During the first cycle of a packet 

22 transmit, the transmit buffer/ controller outputs START on 

23 control lane 0 . Lanes 1 through 7 carry the balance of the 

24 GX Header information. During the last cycle of transmit, 

25 all of the lanes up to the final data byte have control 



Patent Application for High Speed Network Interface by Bechtolsheim, Frazier, and Edsall 17 
File: C:\patent\cisco\13161_MF_Hi_Speed_Int\13161patentdoc Last saved 06/24/99 12:05 AM 



1 signals set to DATA, and data is sent on these data lines. 

2 The following lane has its control and data lines set for 

3 END, and the remaining lanes have their control and data 

4 lines set for IDLE_EVEN, or IDLE_ODD, depending on whether 

5 they occur on an even or odd cycle. Encoders 378a through 

6 378h accept this control signal accompanied by data. During 

7 the times the control input is DATA, this data is encoded 

8 from 8B to 10B as described in the 8B/10B coding scheme of 

9 U.S. Patent #4,486,739. During the times the control input 

10 is CONTROL, the data input is interpreted to produce control 

11 codes such as IDLEJEVEN, 1DLE_0DD, START, and END. The 10B 

12 encoded data is input to transmit serializers 3 86a through 

13 3 86h, and are output as serial data organized by data lanes 

14 3 88a through 3 88h. These outputs are typically fed through 

15 an optical or electrical link to a receive processor. 

16 Transmit clock source 3 68 provides a clock 37 6 for the 

17 transmit buffer/controller 370 and optionally the encoders 

18 378a through 378h at the data rate in use. For a system 

19 sending lOGb/s of data, TX Data 3 64 would clock 8 bytes of 

20 data at a rate of 156.25Mhz into encoders 37 8 and serializer 

21 386 via parallel clock 376. After encoding the data to a 10 

22 bit width 380a, the serializer output stages 386 are clocked 

23 at 1.5625Ghz. It is understood to one skilled in the art 

24 that the actual clocking speed of the interface as provided 

25 by generator 3 68 may be any frequency, as long as the 
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1 serializer clock rate and transmit buffer output clocking 

2 rate are matched in throughput. Additionally, the encoders 

3 may be optionally clocked on input or output, or not at all, 

4 and the serializers may be optionally clocked on input or 

5 output, although best performance may be seen with clocking 

6 at each stage . 

7 Figure 16 shows an alternate transmit processor for the 

8 case where the number of data lanes n = 4, while the TX data 

9 input 404 rate remains constant. Transmit buffer 410 may 

10 accept 8 byte wide inputs on interface 4 04 accompanied by 

11 control/data bits 406, and perform a 2:1 multiplexing to 

12 distribute this input data and control across the 4 data 

13 lane wide interface. The operation of other elements of 

14 figure 16 is the same as the earlier figure 15, with the 

15 exception of clock generator 408, which for a lOGb/s input 

16 rate 404, is now clocking in TX data 404 with a 156.25 input 

17 clock 409, and is using a 312.5Mhz input clock for control 

18 414 and data 412 on lanes 0 to 3 to the encoders 418 and 

19 serializer inputs 426. The serializer 426 outputs are 

20 clocked using a 3.125Ghz clocking rate 424. The output of 

21 this transmit processor is the 4 data lane data stream of 

22 figure 12. As is clear to one skilled in the art, many 

23 other such modification may be made to the transmit 

24 processor, including changing the number of data lanes to 

25 any value of one or more, whereby the level of multiplexing 
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1 of the transmit buffer/controller 410 is related to the 

2 width of the input packet bus divided by the number of data 

3 lanes on the output side of the buffer/controller. For the 

4 case where n=l and a 64 bit packet input is afforded, the 

5 input would be multiplexed 8:1, and only data lane 0 would 

6 be implemented, producing the output described in figure 14. 

7 For n=l, the transmit buffer would clock 8 byte data in at 

8 15 6.2 5Mhz, and send a single stream to the encoder, clocked 

9 into and out of the serializer at 1.25Ghz. The serializer 

10 rate 422 would be 12 . 5Ghz . As indicated earlier, any 

11 clocking rate could be used, as long as the throughputs are 

12 matched between stages, 

13 Figure 17a shows the 8B/10B encoder in further detail. 

14 Data input 412a and control input 414a produce output 42 0a 

15 which is 10B encoded depending on the data and control 

16 inputs, as shown in figure 17b. For the control input 414 

17 set to DATA, line 442 shows 8 bit input data directly 

18 encoded into 10 bit output data according to the method of 

19 patent #4,486,739. Line 440 shows the start transaction, 

20 wherein an 8B START input 412 and the control input 414 set 

21 to CTRL outputs a Start symbol selected from the available 

22 and unused 10B encodings. Similarly, lines 444, 446, 448, 

23 449, and 45 0 show the control input 414 set to CTRL while 

24 END, IDLE_EVEN, IDLE__ODD, IDLE_EVEN_BUSY, and 1 DL E__ODD__BU S Y 

25 are input as 8B data, which causes unique 10B symbols to be 
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1 emitted which are reserved from the available and unused 10B 

2 encodings. There are many different such unique and unused 

3 10B codings, and the best selections are made on the basis 

4 of running disparity and hamming distance from other codes. 

5 In the best mode the codings (in 10B descriptive format) are 

6 START K27.7 

7 END K29.7 

8 IDLE_EVEN K28 . 5 

9 IDLE_ODD K23 .7 

10 IDLE_EVENJBUSY K28.1. 

11 IDLE_ODD_BUSY K2 8.0. 
12 

13 When the receive processor for this end of the link 

14 has a full receive buffer or is no longer capable of 

15 receiving additional data, it signals this to the 

16 transmitter on the opposite end of the link by emitting 

17 EVEN_IDLE_BUSY or ODD_.IDLE_.BUSY, as shown in lines 449 and 

18 450. 

19 Figure 18 shows the receive processor 452. Input data 

20 is applied as serial streams organized by data lane, shown 

21 as 450a through 450h. This data is applied to the receive 

22 deserializers 454a through 454h, which output 10 bit wide 

23 streams of data 458a through 458h, organized by data lane. 

24 These 10B streams are applied to the 10B/8B decoders 456a 

25 through 456h, which decode 8 bit data 464a through 464h and 
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1 control information 466a through 466h, shown for lane 0 as 

2 464a and 466a respectively, and repeated identically for all 

3 decoders 456a through 456h. Receive elasticity 

4 buffer/controller 468 accepts these inputs and organizes 

5 packet data to transfer to output port 474 based on 

6 receiving a control input 466a asserted with the data input 

7 464a having the value START on lane 0, and header data on 

8 the remaining lanes 464b and 466a through 464h through 466h, 

9 followed by the data packet, followed by the control signal 

10 466 asserted accompanied by data 464 having the value END on 

11 the appropriate data lane. Thereafter IDLE signals are 

12 received. The controller is able to extract a variety of 

13 error conditions, such as START appearing on a data lane 

14 other than 0, an improperly formed IDLE_EVEN, IDLE_ODD, 

15 IDLE„EVEN_BUSY, IDLE_ODD_BUSY or END transaction, and the 

16 like. These error conditions are signaled through error 

17 output 47 6. Each deserializer may locally recover a clock 

18 459 for use in clocking the decoder 456. The receive 

19 elasticity buffer/controller also serves to isolate the 

20 clock domains between the input clocking rate controlled by 

21 the receive data speed whereby control 46 6 and data 464 

22 signals enter the controller at rates determined by the link 

23 far-end sender and buffered data 474 is clocked out of the 

24 receive elasticity buffer by local system clock 472. While 

25 the receive clock domain boundary is shown in the elasticity 
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1 buffer 468, it is clear to one skilled in the art that this 

2 boundary could be located anywhere in the receive processor. 

3 In the example shown in figure 18, data is clocked into 

4 decoders 456a-456h by recovered clock 459, which is locked 

5 to the frequency of the sending source using clock recovery 

6 methods well known in the art. Data is clocked out of the 

7 receive buffer by local clock 462, which is derived from 

8 system clock 472. 

9 Figure 19 shows the case where the receive processor 

10 has the number of data lanes n = 4. This operates 

11 analogously to the receive processor of figure 18, except 

12 that the data rate of data presented to the receive 

13 buffer/controller on inputs 494 and 496 is doubled. 

14 Figure 20a shows the 10B/8B decoder. 10 bit parallel 

15 input 458a is converted to 8 bit output data 464a, and 1 bit 

16 control 466a. Figure 20b shows the cases 470, 474, 476, 

17 478, 480, and 482 for Start, End, Even_Idle, Odd_Idle, 

18 Even„Idle_Busy , and Odd_Idle_Busy respectively. Case 472 

19 shows 10B input data decoded into 8B output data, and the 

20 control output signal asserted as type DATA. 

21 In the manner described, data packets may be furnished 

22 to a controller, encoded into data lanes, serialized, and 

23 transmitted as clock-encoded data. These serial streams of 

24 byte-organized data may remotely deserialized, decoded, and 

25 re-assembled back into data packets for handling. 
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1 Additionally, header information may optionally be pre- 

2 pended such that information required by the receiving 

3 equipment may be easily retrieved during the interval the 

4 data packet is being received. The above description of the 

5 high speed transmit and receive processors has been drawn to 

6 a particular implementation to afford a complete 

7 understanding of the invention as practiced in its best 

8 mode. Although the invention has been described through the 

9 best mode case, it is clear to one skilled in the art that 

10 there are many other ways of implementing the described 

11 invention. For example, while the figures describe receive 

12 and transmit processors having n=l to n-8 data lanes, it is 

13 clear that an arbitrary number of channels would operate in 

14 the same manner, and that the parallel paths and codings do 

15 not need to be based on 8-bit bytes. Further, while the 

16 encoding and decoding of symbols is based on the standard 

17 method of 8B/10B coding of U.S. Patent #4,486,739, other 

18 coding/decoding methods could be used. The transmit 

19 buffer/controller is shown as generating control signals 

20 which indicate the time that particular codes are to be 

21 emitted by the encoders, functions which could be performed 

22 by a separate controller. While the clocking rates have 

23 been chosen to illustrate a 10Gb/ s data rate for exemplar 

24 purposes, no such clock speed limitation should be inferred 
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1 in the present invention, which offers the advantages 

2 described for any data rate or clocking speed. 
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I We claim: 
2 

3 1) A process for transmitting a packet having a header 

4 and variable length payload on a communications interface 

5 comprising the steps: 

6 a first step of sending IDLE symbols until a 

7 synchronization time has passed; 

8 a second step of sending said packet including a START 

9 symbol and TYPE field identifying the format of said payload 
10 including an FCS sequence; 

II a third step of sending said variable length payload; 

12 a fourth step of sending a terminator including an END 

13 symbol indicating end of transmission of said packet; 

14 a fifth step of sending IDLE symbols if next said 

15 packet is not ready to transmit, or returning to said second 

16 step if said next packet is ready to transmit. 
17 

18 2) The process of claim 1 wherein said TYPE field 

19 uniquely identifies said payload format, said format 

20 including Ethernet packets, native IP packets, ATM cells, 

21 and control packets . 

22 

23 3 ) the process of claim 2 wherein said header further 

24 includes declaration fields for at least one of BPDU, 

25 PRIORITY, VLAN__ID, and an application specific field. 
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1 



2 4) The process of claim 3 wherein said BPDU field is 1 

3 bit in size. 

4 

5 5) The process of claim 3 wherein said PRIORITY field 

6 is 3 bits in size. 

7 

8 6) The process of claim 3 wherein said VLAN_ID field is 

9 12 bits in size. 

10 

11 7) The process of claim 3 wherein said application 

12 specific field is 32 bits in size. 

13 

14 8) The process of claim 3 wherein said header 

15 comprises, in sequence, said START symbol, said BPDU field, 

16 said TYPE field, said PRIORITY field, said VLAN_ID field, 

17 and said application-specific field. 

18 

19 9) The process of claim 1 wherein a plurality n of data 

20 lanes carry said header, said payload, and said END symbol. 

21 

22 10) the process of claim 9 wherein 
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1 said second step comprises transmitting said header 

2 across said n data lanes until all said header information 

3 has been sent; 

4 said third step comprises transmitting said variable 

5 length payload, wherein during a final payload cycle, said 

6 payload ends on said data lane m; 

7 for the case where m < n, said fourth step includes 

8 sending on said final payload cycle said END symbol on lane 

9 m+1, and said IDLE symbol on any available data lanes rn=n+2 

10 through n; 

11 for the case where m = n, said fourth step comprises 

12 sending said END symbol on said data lane 0, and said IDLE 

13 symbol on said data lane 1 through said data lane n. 

14 

15 11) The process of claim 10 where n = 8. 

16 

17 12) The process of claim 10 where n = 4. 

18 

19 13) The process of claim 10 where n = 2. 

20 

21 14) the process of claim 9 where n = 1, and 
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1 said second step comprises transmitting said header on 

2 said data lane until all said header information has been 

3 sent; 

4 said third step comprises transmitting said variable 

5 length payload on said data lane, 

6 said fourth step comprises sending said END symbol on 

7 said data lane. 

8 

9 15) The process of claim 10 wherein at least one said 

10 data lane comprises a serial electrical link. 

11 

12 16) The process of claim 10 wherein at least one said 

13 data lane comprises a parallel electrical link. 

14 

15 17) The process of claim 10 wherein at least one said 

16 data lane comprises one or more serial or parallel optical 

17 links. 

18 

19 18) The process of claim 10 wherein said first step 

20 comprises the transmission of said IDLE symbols on all said 

21 n data lanes . 



22 
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1 19) The process of claim 18 wherein said IDLE symbols 

2 are transmitted across all said n data lanes when there is 

3 no said packet data available to transmit. 



4 

5 20) The process of claim 19 wherein successive data 

6 lane cycles toggle successively between the states odd and 

7 even . 

8 

9 21) The process of claim 2 0 wherein said IDLE symbols 

10 transmitted comprise IDLE_ODD symbols during said odd state, 

11 and IDLE_EVEN symbols during said even state. 



12 

13 22) A communication interface comprising n data lanes, 

14 said interface sequentially transmitting a header 

15 distributed across a plurality of said data lanes, a 

16 variable amount of payload data distributed across a 

17 plurality of said n data lanes. 
18 



19 23) The communication interface of claim 22 wherein 

20 said transmission of said header includes transmitting a 

21 START symbol on first said data lane, and the transmission 

22 of said payload data is followed by an END symbol on at 

23 least one said data lane. 



24 
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1 24) The communication interface of claim 23 wherein 

2 said transmission of said payload data includes transmitting 

3 data across said n data lanes up to data lane m, where m <= 

4 n. 
5 

6 25) The communication interface of claim 24 wherein if 

7 said m < n, said END symbol is transmitted on data lane m+1, 

8 and if said m=n, said END symbol is transmitted on data lane 

9 0. 
10 

11 26) The communication interface of claim 2 5 wherein 

12 each said data lane is identified by the alternating states 

13 of odd and even cycles. 
14 

15 27) The communication interface of claim 2 6 wherein 

16 said IDLE symbol is IDLE_EVEN during said even cycle and 

17 IDLE_ODD during said odd cycle. 
18 

19 28) The communication interface of claim 27 wherein all 

20 said data lane 0 through data lane n transmit IDLE_EVEN 

21 during said even cycles, and IDLE_0DD during said odd 

22 cycles. 
23 
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1 29) The communication interface of claim 28 where 

2 IDLE_EVEN or IDLE_ODD are transmitted after said END symbol 

3 at least once during every interval t e lasticity * 
4 

5 30) The communication interface of claim 29 where 

6 t e i as ticity ~ T transmit * clk_Offset, 

7 where 

8 T transmit = time since last IDLE transmittal 

9 clk_offset = (maximum Transmit clock rate - minimum 

10 receive clock rate) / (minimum receive clock rate) . 
11 

12 31) A transmit processor comprising: 

13 a busy input; 

14 a transmit buffer/controller accepting packet data 

15 comprising a header and a payload as input, arranging said 

16 packet data into a plurality n of data lanes, and delivering 

17 to each said data lane unencoded transmit data and a control 

18 signal, whereby when said control signal is asserted, said 

19 unencoded transmit data includes at least one of the values 

20 START, END, IDLE, IDLE_BUSY and when said control signal is 

21 not asserted, said transmit data includes said packet data; 

22 a plurality n of transmit encoders, each having an 

23 input and an output, each of said transmit encoder inputs 

24 uniquely coupled to one of said transmit buff er /controller 

25 data lanes, said transmit encoder input comprising said 
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1 unencoded transmit data and said control signal, said 

2 transmit encoder output producing a unique encoded output 

3 value for each said unencoded transmit data value when said 

4 control signal is not asserted, and producing a unique 

5 encoded output values for each unencoded transmit data 

6 START, END, IDLE, and IDLE_BUSY when said control signal is 

7 asserted; 

8 a plurality n of transmit serializers, each having an 

9 input uniquely coupled to one of said transmit encoder 

10 outputs, said transmit serializers outputting a single 

11 serial stream of data from said transmit serializer input; 

12 wherein said transmit buff er /controller sends said 

13 header by outputting on said first data lane the asserted 

14 said control and said unencoded transmit data START, and 

15 simultaneously outputs the remainder of said header on said 

16 remaining data lanes accompanied by said unasserted control 

17 signal for each said data lane, 

18 thereafter and on each successive cycle said transmit 

19 buffer/controller distributes said payload data on all said 

20 data lanes and sends it to said transmit encoder with said 

21 unasserted control signal accompanied by said payload data, 

22 until unsent said payload data can not fully span said n 

23 data lanes, 

24 thereafter said transmit buffer/controller sends the 

25 last said payload data on each said data lane with 



Patent Application for High Speed Network Interface by Bechtolsheim, Frazier, and Edsall 33 
File: C:\patent\cisco\13161_MF_Hi_Speed_Int\13161patentdoc Last saved 06/24/99 12:05 AM 



1 associated said control signal unasserted, with following 

2 said data lane having said control signal asserted 

3 accompanied by said unencoded data END, and the remaining 

4 said data lanes having said control signal asserted 

5 accompanied by said unencoded data IDLE. 
6 

7 32) The transmit processor of claim 31 wherein each 

8 said transmit cycle has the state odd or even, and said IDLE 

9 comprises an IDLE_EVEN sent on said even cycles or an 
10 IDLE_ODD sent on said odd cycle. 

11 

12 33) The transmit processor of claim 32 wherein each 

13 successive transmit cycle alternates between odd or even, 

14 said IDLE_EVEN is sent during even cycles, and IDLE_0DD is 

15 sent during odd cycles. 
16 

17 34) The transmit processor of claim 32 wherein said 

18 IDLE comprises an IDLE when said busy input is not asserted, 

19 or a IDLE_BUSY when said busy input is asserted. 
20 

21 35) The transmit processor of claim 34 wherein said 

22 IDLE comprises an IDLE_EVEN_BUSY during said even cycle when 

23 said busy input is asserted, an IDLE_EVEN during said even 

24 cycle when said busy input is not asserted, an I DLE_ODD_BU S Y 

25 during said odd cycle when said busy input is asserted, and 
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1 an IDLE_ODD during said odd cycle when said busy is not 

2 asserted. 
3 

4 36) The transmit processor of claim 35 wherein said 

5 transmit encoder comprises an 8B/10B encoder. 
6 

7 37) The transmit processor of claim 3 6 wherein the 

8 number of said data lanes n = 8 . 
9 

10 38) The transmit processor of claim 3 6 wherein the 

11 number of said data lanes n = 4. 
12 

13 39) The transmit processor of claim 3 6 wherein the 

14 number of said data lanes n = 2 . 
15 

16 40) The transmit processor of claim 3 6 wherein the 

17 number of said data lanes n = 1. 
18 

19 41) The transmit processor of claim 3 6 wherein the 10B 

20 coding value for symbol START is K27.7. 
21 

22 42) The transmit processor of claim 3 6 wherein the 10B 

23 coding value for symbol END is K29.7. 



24 
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1 43) The transmit processor of claim 3 6 wherein the 10B 

2 coding value for symbol IDLE_EVEN is K28.5. 
3 

4 44) The transmit processor of claim 3 6 wherein the 10B 

5 coding value for symbol IDLE_ODD is K23.7. 
6 

7 45) The transmit processor of claim 3 6 wherein the 10B 

8 coding value for symbol I DLE_E VEN_BU S Y is K28.1. 
9 

10 46) The transmit processor of claim 3 6 wherein the 10B 

11 coding value for symbol I DLE_ODD_BUS Y is K28.0. 
12 

13 47) The transmit processor of claim 3 6 wherein the 10B 

14 coding values for the symbols START, END, IDLE__EVEN, 

15 IDLE_EVEN_BUSY, IDLE__ODD , and IDLE_ODD_BUSY have unique 

16 values when compared to any coded 10B data value. 



17 

18 48) The transmit processor of claim 47 wherein the 10B 

19 coding values for the symbols START, END, IDLE_EVEN, 

20 IDLE_EVEN_BUSY , IDLE_ODD, and IDLE_ODD_BUSY are separated by 

21 hamming distance 2 . 
22 

23 49) A receive processor comprising: 
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1 a plurality n of receive deserializers each accepting 

2 as input a serial stream of encoded data and outputting 

3 deserialized encoded data; 

4 a plurality n of receive decoders each uniquely coupled 

5 to and accepting as input said deserialized encoded data and 

6 providing as output decoded data and decoded control 

7 signals, said decoded data including at least on of the 

8 values START, END, and IDLE when said control signal is 

9 asserted; 

10 a receive buffer/controller for the formation of data 

11 packets, said buffer/controller having a plurality n of 

12 inputs, each uniquely coupled to said decoded data and said 

13 decoded control, said buffer/controller having a busy output 

14 and a data output, said receive buff er /controller awaiting 

15 START on said first lane with associated control signal 

16 asserted, and storing a header on the remaining said data 

17 lanes when said START is received, and transferring to said 

18 data output all subsequent data while said control signal is 

19 unasserted for all said data lanes, and upon receipt of said 

20 END accompanied by the assertion of said associated control 

21 signal on any data lane, transferring said decoded data to 

22 said data output all said received data up to but not 

23 including said data lane having said control signal END. 
24 
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1 50) The receive processor of claim 49 wherein said IDLE 

2 comprises the symbols IDLE_EVEN, IDLE_0DD, IDLE_EVEN_BUSY, 

3 and IDLE„EVEN_ODD . 
4 

5 51) The receive processor of claim 50 including a busy 

6 signal wherein the reception of I DL E_EVEN_BU S Y or 

7 IDLE_ODD_JBUSY causes said receive processor to assert said 

8 busy output . 
9 

10 52) The receive processor of claim 49 wherein said 

11 receive decoder uses a 10B/8B decoding method for converting 

12 said encoded data into said decoded data. 
13 

14 53) The receive processor of claim 52 wherein each 

15 receive deserializer achieves synchronization using the 

16 symbols IDLE„EVEN and IDLE_0DD. 
17 

18 54) the receive processor of claim 52 wherein the 10B 

19 coding value for symbol START is K27.7. 
20 

21 55) The receive processor of claim 52 wherein the 10B 

22 coding value for symbol END is K29.7. 
23 

24 56) The receive processor of claim 52 wherein the 10B 

25 coding value for symbol IDLE_EVEN is K28.5. 
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1 

2 57) The receive processor of claim 52 wherein the 10B 

3 coding value for symbol 1DLE_0DD is K23.7. 
4 

5 58) The receive processor of claim 52 wherein the 10B 

6 coding value for symbol I DL E_E VEN_BU S Y is K2 8.1. 
7 

8 59) The receive processor of claim 52 wherein the 10B 

9 coding value for symbol IDLE_ODD_BUSY is K2 8.0. 



10 
11 



12 60) The receive processor of claim 52 wherein the 10B 

13 encoded values for the symbols START , END, IDLE_EVEN and 

14 1DLE_0DD have unique values when compared to any other 

15 encoded 10B data value. 
16 

17 61) The receive processor of claim 60 wherein the 10B 

18 coding values for the symbols START, END, IDLE_EVEN, and 

19 IDLE_ODD are separated by hamming distance 2 . 



20 



21 62) The receive processor of claim 49 wherein the 

22 number of data lanes n = 8. 
23 

24 63) The receive processor of claim 49 wherein the 

25 number of data lanes n - 4. 
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1 

2 64) The receive processor of claim 49 wherein the 

3 number of data lanes n = 2 . 
4 

5 65) The receive processor of claim 49 wherein the number of 

6 data lanes n - 1 . 
7 

8 66) A communications interface for sending or receiving 

9 a packet, said packet comprising, in sequence, a header, 

10 variable length payload, and a terminator; 

11 said header including a START symbol and a TYPE field 

12 identifying the format of said payload; 

13 said terminator including an END symbol; 

14 wherein said START symbol is transmitted first, 

15 followed by the remainder of said header, followed by said 

16 variable length packet data, followed by said terminator. 
17 

18 67) The interface of claim 6 6 wherein said TYPE field 

19 uniquely identifies said payload format, said format 

20 including Ethernet packets, ATM cells, and control packets. 

21 

22 68) the interface of claim 67 wherein said header 

23 further includes declaration fields for at least one of 

24 BPDU, PRIORITY, VLAN_ID, and an application specific field. 



25 
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1 69) The interface of 

2 is 1 bit in size. 

3 

4 70) The interface of 

5 field is 3 bits in size. 

6 

7 71) The interface of 

8 field is 12 bits in size. 

9 

10 72) The interface of 

11 specific field is 32 bits 



claim 68 wherein said BPDU field 

claim 69 wherein said PRIORITY 

claim 7 0 wherein said VLAN_ID 

claim 71 wherein said application 
in size. 
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1 ABSTRACT 

2 A high speed communications interface divides data into a 

3 plurality of lanes, each lane encoded with clocking 

4 information, serialized, and sent to an interface. During 

5 cycles when there is no available data to send, IDLE_EVEN 

6 and IDLE_ODD cells are sent on alternating cycles. Data is 

7 transmitted by sending a header which spans all lanes and 

8 includes a START symbol. The final data transaction 

9 includes a Frame Check Sequence (FCS) which operates over 

10 the entire header and data. The packet is terminated by an 

11 END symbol, which is sent after the final data, and the 

12 remainder of the lanes are padded with IDLE„EVEN, 1DLE_0DD , 

13 IDLE__EVEN_BUSY , or IDLE_ODD_BUSY cycles. The interface has 

14 a variable clock rate. 
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